Learning Sparse High Dimensional Filters: Image Filtering, Dense CRFs and Bilateral Neural Networks

Bilateral filters have wide spread use due to their edge-preserving properties. The common use case is to manually choose a parametric filter type, usually a Gaussian filter. In this paper, we will generalize the parametrization and in particular derive a gradient descent algorithm so the filter parameters can be learned from data. This derivation allows to learn high dimensional linear filters that operate in sparsely populated feature spaces. We build on the permutohedral lattice construction for efficient filtering. The ability to learn more general forms of high-dimensional filters can be used in several diverse applications. First, we demonstrate the use in applications where single filter applications are desired for runtime reasons. Further, we show how this algorithm can be used to learn the pairwise potentials in densely connected conditional random fields and apply these to different image segmentation tasks. Finally, we introduce layers of bilateral filters in CNNs and propose bilateral neural networks for the use of high-dimensional sparse data. This view provides new ways to encode model structure into network architectures. A diverse set of experiments empirically validates the usage of general forms of filters

Semantic Instance Annotation of Street Scenes by 3D to 2D Label Transfer

Semantic annotations are vital for training models for object recognition, semantic segmentation or scene understanding. Unfortunately, pixelwise annotation of images at very large scale is labor-intensive and only little labeled data is available, particularly at instance level and for street scenes. In this paper, we propose to tackle this problem by lifting the semantic instance labeling task from 2D into 3D. Given reconstructions from stereo or laser data, we annotate static 3D scene elements with rough bounding primitives and develop a model which transfers this information into the image domain. We leverage our method to obtain 2D labels for a novel suburban video dataset which we have collected, resulting in 400k semantic and instance image annotations. A comparison of our method to state-of-the-art label transfer baselines reveals that 3D information enables more efficient annotation while at the same time resulting in improved accuracy and time-coherent labels.Comment: 10 pages in Conference on Computer Vision and Pattern Recognition (CVPR), 201

Quasi-Newton Methods: A New Direction

Four decades after their invention, quasi-Newton methods are still state of the art in unconstrained numerical optimization. Although not usually interpreted thus, these are learning algorithms that fit a local quadratic approximation to the objective function. We show that many, including the most popular, quasi-Newton methods can be interpreted as approximations of Bayesian linear regression under varying prior assumptions. This new notion elucidates some shortcomings of classical algorithms, and lights the way to a novel nonparametric quasi-Newton method, which is able to make more efficient use of available information at computational cost similar to its predecessors.Comment: ICML201

Stochastic Nonlinear Model Predictive Control with Guaranteed Error Bounds Using Compactly Supported Wavelets

In model predictive control, a high quality of control can only be achieved, if the model of the system reflects the real-world process as precisely as possible. Therefore, the controller should be capable of both handling a nonlinear system description and systematically incorporating uncertainties affecting the system. Since stochastic nonlinear model predictive control (SNMPC) problems in general cannot be solved in closed form, either the system model or the occurring densities have to be approximated. In this paper, we present an SNMPC framework, which approximates the densities and the reward function by their wavelet expansions. Due to the few requirements on the shape and family of the densities or reward function, the presented technique can be applied to a large class of SNMPC problems. For accelerating the optimization, we additionally present a novel thresholding technique, the so-called dynamic thresholding, which neglects coefficients that are insignificant, while at the same time guaranteeing that the optimal control input is still chosen. The capabilities of the proposed approach are demonstrated by simulations with a path planning scenario

Permutohedral Lattice CNNs

This paper presents a convolutional layer that is able to process sparse input features. As an example, for image recognition problems this allows an efficient filtering of signals that do not lie on a dense grid (like pixel position), but of more general features (such as color values). The presented algorithm makes use of the permutohedral lattice data structure. The permutohedral lattice was introduced to efficiently implement a bilateral filter, a commonly used image processing operation. Its use allows for a generalization of the convolution type found in current (spatial) convolutional network architectures

Nonlinear Bayesian Estimation with Compactly Supported Wavelets

Bayesian estimation for nonlinear systems is still a challenging problem, as in general the type of the true probability density changes and the complexity increases over time. Hence, approximations of the occurring equations and/or of the underlying probability density functions are inevitable. In this paper, we propose an approximation of the conditional densities by wavelet expansions. This kind of representation allows a sparse set of characterizing coefficients, especially for smooth or piecewise smooth density functions. Besides its good approximation properties, fast algorithms operating on sparse vectors are applicable and thus, a good trade-off between approximation quality and run-time can be achieved. Moreover, due to its highly generic nature, it can be applied to a large class of nonlinear systems with a high modeling accuracy. In particular, the noise acting upon the system can be modeled by an arbitrary probability distribution and can influence the system in any way

Novel strategies for the synthesis of unsymmetrical glycosyl disulfides

yesNovel strategies for the efficient synthesis of unsymmetrical glycosyl disulfides are reported. Glycosyl disulfides are increasingly important as glycomimetics and molecular probes in glycobiology. Sialosyl disulfides are synthesised directly from the chlorosialoside Neu5Ac2Cl, proceeding via a thiol-disulfide exchange reaction between the sialosyl thiolate and symmetrical disulfides. This methodology was adapted and found to be successfully applicable to the synthesis of unsymmetrical glucosyl disulfides under mild conditions

Learning an event sequence embedding for event-based deep stereo

Today, a frame-based camera is the sensor of choice for machine vision applications. However, these cameras, originally developed for acquisition of static images rather than for sensing of dynamic uncontrolled visual environments, suffer from high power consumption, data rate, latency and low dynamic range. An event-based image sensor addresses these drawbacks by mimicking a biological retina. Instead of measuring the intensity of every pixel in a fixed time-interval, it reports events of significant pixel intensity changes. Every such event is represented by its position, sign of change, and timestamp, accurate to the microsecond. Asynchronous event sequences require special handling, since traditional algorithms work only with synchronous, spatially gridded data. To address this problem we introduce a new module for event sequence embedding, for use in difference applications. The module builds a representation of an event sequence by firstly aggregating information locally across time, using a novel fully-connected layer for an irregularly sampled continuous domain, and then across discrete spatial domain. Based on this module, we design a deep learning-based stereo method for event-based cameras. The proposed method is the first learning-based stereo method for an event-based camera and the only method that produces dense results. We show that large performance increases on the Multi Vehicle Stereo Event Camera Dataset (MVSEC), which became the standard set for benchmarking of event-based stereo methods